As consumer computing devices continue to evolve, depth camera systems are increasingly being included in these devices to provide additional versatility and enjoyment for consumers. Depth camera systems, such as structured light systems (including a camera and a projector), utilize stereo matching techniques to triangulate depth in visual content items by solving a correspondence between pixels in a visual content item observed by the camera and pixels representing reference light patterns projected by the projector. For instance, patches of pixels in a visual content item observed by the camera can be matched to corresponding pixels in a visual content item representing the reference light pattern projected by the projector. Depth can be derived for each corresponding pixel based on triangulation and epipolar geometric relationships between the matched pixels. While more efficient processing techniques are continually being implemented for matching pixels, depth camera systems suffer from high computational costs associated with matching pixels between the camera and projector.